A New Approach to Simulation Studies in Computerized Adaptive Testing

Precision of trait estimation and determination of item exposure rates are always important considerations in conducting CATs. To date, simulation techniques have been used in research concerning these two considerations. Even though conducting simulation studies has some advantages over using real human subjects, the simulation results are not as accurate as analytically obtained results. To improve the accuracy of the results, and to avoid time consuming simulation work, more research on analytical solutions to problems is needed. The purpose of this study was to investigate the precision of CAT trait estimation and to determine CAT item exposure rates using an analytical approach (referred to as Tree). The results thus obtained were compared to those obtained using a simulation study, where three different levels of replications, 100, 500, and 1,000(referred to as S100, S500, and S1K respectively) were implemented. Based on this study, the differences among S100, S500, S1K, and Tree decreased as test length increased. For test length as long as 20 items, the differences among S500, S1K, and Tree were negligible. In other words, a simulation study with 500 replications can provide results as accurate as the analytical approach for a 20-item test.